import TabItem from "@theme/TabItem";
import Tabs from "@theme/Tabs";

import ComponentConfiguration from "@site/src/pages/components-explorer/_components/ComponentConfiguration";
import ComponentHeader from "@site/src/pages/components-explorer/_components/ComponentHeader";
import ComponentTroubleshooting from "@site/src/pages/components-explorer/_components/ComponentTroubleshooting/index.mdx";
import Camera from "@site/src/pages/components-explorer/_domains/camera/index.mdx";

import ComponentMetadata from "./_meta";
import config from "./config.json";

<ComponentHeader meta={ComponentMetadata} />

FFmpeg enables Viseron to read frames from cameras.

FFmpeg can be quite complex to work with, but in most cases the automatically generated commands will fit your needs.

[Hardware acceleration](#hardware-acceleration) is available on a wide variety of systems.

## Configuration

<details>
  <summary>Configuration example</summary>

```yaml title="/config/config.yaml"
ffmpeg:
  camera:
    camera_one:
      name: Camera 1
      host: 192.168.XX.X
      port: 554
      path: /Streaming/Channels/101/
      username: !secret camera_one_user
      password: !secret camera_one_pass
      substream:
        path: /Streaming/Channels/102/
        stream_format: rtsp
        port: 554
      mjpeg_streams:
        my_stream:
          width: 100
          height: 100
          draw_objects: true
          rotate: 45
          mirror: true
        objects:
          draw_objects: true
          draw_zones: true
          draw_motion: true
          draw_motion_mask: true
          draw_object_mask: true
      recorder:
        idle_timeout: 5
      frame_timeout: 10
```

</details>

<ComponentConfiguration config={config} />

<Camera />

### Hardware acceleration

Viseron supports both hardware accelerated decoding and encoding.<br />
This means your CPU will be offloaded and give you a significant performance increase.

Supported hardware:

- NVIDIA GPU
- VA-API on compatible Intel CPU's
- Raspberry Pi 3
- Raspberry Pi 4
- NVIDIA Jetson Nano

Viseron will detect what system you are running and will automagically utilize the hardware acceleration.

:::warning

The Jetson Nano support is very limited in FFmpeg. If you have a Nano i suggest looking at the `gstreamer` component instead.

:::

### Default FFmpeg decoder command

A default FFmpeg decoder command is generated.
To use hardware acceleration, the command varies a bit depending on the Docker container you use.

<details open>
<summary>Commands</summary>

<Tabs groupId="hwaccel">
<TabItem value="nvidia" label="NVIDIA">

NVIDIA GPU support in the <b>roflcoopter/amd64-cuda-viseron</b> image:

```
ffmpeg -hide_banner -loglevel error -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -c:v h264_cuvid -rtsp_transport tcp -i rtsp://{username}:{password}@{host}:{port}{path} -f rawvideo -pix_fmt nv12 pipe:1
```

</TabItem>
<TabItem value="vaapi" label="VA-API">

VAAPI support in the <b>roflcoopter/viseron</b> image:

```
ffmpeg -hide_banner -loglevel error -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -rtsp_transport tcp -i rtsp://{username}:{password}@{host}:{port}{path} -f rawvideo -pix_fmt nv12 pipe:1
```

</TabItem>
<TabItem value="rpi3" label="RPi3">

For RPi3 in the <b>roflcoopter/rpi3-viseron</b> image:

```
ffmpeg -hide_banner -loglevel error -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -stimeout 5000000 -use_wallclock_as_timestamps 1 -vsync 0 -c:v h264_mmal -rtsp_transport tcp -i rtsp://{username}:{password}@{host}:{port}{path} -f rawvideo -pix_fmt nv12 pipe:1
```

</TabItem>
</Tabs>

</details>

This means that you do **not** have to set `hwaccel_args` _unless_ you have a specific need to change the default command (say you need to change `h264_cuvid` to `hevc_cuvid`)

### Custom FFmpeg decoder command

You can customize the generated command through the config.
It can be a bit hard to get this right so it is not recommended unless you know what you are doing.

The command is built up like this:

```python
"ffmpeg" + global_args + input_args + hwaccel_args + codec + "-rtsp_transport tcp -i " + (stream url) + " -vf " + video_filters + output_args
```

Each entry in `video_filters` are appended together, separated with a `,`.

<details>
  <summary>Config example to rotate image 180 degrees</summary>

```yaml title="/config/config.yaml"
ffmpeg:
  camera:
    camera_1:
    ....
      video_filters:   # These filters rotate the images processed by Viseron
        - transpose=2
        - transpose=2
      recorder:
        video_filters:   # These filters rotate the recorded video
          - transpose=2
          - transpose=2
```

And the resulting command looks like this:

```shell
ffmpeg -hide_banner -loglevel error -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts -use_wallclock_as_timestamps 1 -vsync 0 -stimeout 5000000 -c:v h264_cuvid -rtsp_transport tcp -i rtsp://*****:*****@192.168.**.**:554/Streaming/Channels/101/ -vf transpose=2,transpose=2,fps=1.0 -f rawvideo -pix_fmt nv12 pipe:1
```

</details>

### FFprobe Stream Information

Viseron needs to know the width, height, FPS and audio/video codecs of your stream.<br />
FFprobe is used on initialization to figure all this information out.

Some cameras dont play nice with this and fail to report some information.<br />
To circumvent this you can manually specify the stream information.

#### FFprobe timeout

Sometimes FFprobe fails to connect to the stream and times out.<br />
If this is a recurring issue you should specify all of `width`, `height`, `fps`, `codec` and `audio_codec` manually.
Viseron will then not need to call FFprobe and startup will be significantly faster.

### Recoverable Errors

Sometimes FFmpeg prints errors which are not fatal, such as `[h264 @ 0x55b1e115d400] error while decoding MB 0 12, bytestream 114567`.<br />
Viseron always performs a sanity check on the FFmpeg decoder command with `-loglevel fatal`.<br />
If Viseron gets stuck on an error that you believe is **not** fatal, you can add a subset of that error to `ffmpeg_recoverable_errors`.<br />
So to ignore the error above you would add this to your configuration:

```yaml
ffmpeg_recoverable_errors:
  - error while decoding MB
```

### Recorder

[FFmpeg segments](https://www.ffmpeg.org/ffmpeg-formats.html#segment_002c-stream_005fsegment_002c-ssegment) are used to handle recordings.<br />
FFmpeg will write small 5 second segments of the stream to disk.<br />

The reason for using segments instead of just starting the recorder on an event, is to support the `lookback` feature which makes it possible to record _before_ an event actually happened.
It also makes it possible to have continuous recordings.

### Store video segments in memory

To place the video segments in memory instead of writing to disk, you can mount a tmpfs disk in the container.
This will use more memory but reduce the load on your harddrives.

<details>
  <summary>Example tmpfs configuration</summary>

Example Docker command

```shell
docker run --rm \
  -v {segments path}:/segments \
  -v {snapshots path}:/snapshots \
  -v {thumbnails path}:/thumbnails \
  -v {event clips path}:/event_clips \
  -v {timelapse path}:/timelapse \
  -v {config path}:/config \
  -v /etc/localtime:/etc/localtime:ro \
  -p 8888:8888 \
  --tmpfs /tmp/tier1 \
  --name viseron \
  --shm-size=1024mb \
  roflcoopter/viseron:latest
```

Example docker-compose

```yaml
services:
  viseron:
    image: roflcoopter/viseron:latest
    container_name: viseron
    shm_size: "1024mb"
    volumes:
      - {segments path}:/segments
      - {snapshots path}:/snapshots
      - {thumbnails path}:/thumbnails
      - {event clips path}:/event_clips
      - {timelapse path}:/timelapse
      - {config path}:/config
      - /etc/localtime:/etc/localtime:ro
    ports:
      - 8888:8888
    tmpfs:
      - /tmp/tier1
```

```yaml title="/config/config.yaml"
storage:
  recorder:
    tiers:
      # Store 50 MB of segments in RAM disk
      - path: /tmp/tier1
        move_on_shutdown: true # Important to not lose segments on shutdown
        events:
          max_size:
            mb: 50
      # Keep 50 GB of segments on a normal drive
      - path: /config/tier2
        events:
          max_size:
            gb: 50
```

</details>

### Substream

Using the substream is a great way to reduce the system load from FFmpeg.<br />
When configured, two FFmpeg processes will spawn:<br />

- One that reads the main stream and creates segments for recordings. Codec `-c:v copy` is used so practically no resources are used.<br />
- One that reads the substream and pipes frames to Viseron for motion/object detection.

To really benefit from this you should reduce the framerate of the substream to match the lowest fps set for either motion or object detection.<br />
It is also a good idea to change the resolution to something lower than the main stream.

### Rotating video

If you rotate your camera 90 or 180 degrees, you can rotate the video in Viseron to match.<br />
To do this you can use the `video_filters` option in the config.<br />

:::note

If you are rotating the video 90 degrees, you need to tell Viseron the width and height of the video, which should be the opposite of the cameras real resolution.
If you have a camera with 1920x1080 resolution, you need to set `width: 1080` and `height: 1920` in the config.

:::

<details>
  <summary>Config to rotate 90 degrees clockwise</summary>

```yaml title="/config/config.yaml"
ffmpeg:
  camera:
    camera_one:
      name: Camera 1
      host: 192.168.XX.X
      port: 554
      path: /Streaming/Channels/101/
      username: !secret camera_one_user
      password: !secret camera_one_pass
      // highlight-start
      video_filters: # Rotate the frames processed by Viseron
        - transpose=1
      width: 1080 # Width of the rotated video = height of the camera
      height: 1920 # Height of the rotated video = width of the camera
      recorder:
        idle_timeout: 5
        video_filters: # Rotate the recorded videos
          - transpose=1
      // highlight-end
```

</details>

### Adding timestamp to video

To add a timestamp to the video you can use the `drawtext` filter in FFmpeg.<br />
This filter is quite complex and you can read more about it [here](https://ffmpeg.org/ffmpeg-filters.html#drawtext-1).

To apply the filter you need to add it to the `video_filters` option in the config.

<details>
  <summary>Config to add a timestamp to the video</summary>

```yaml title="/config/config.yaml"

ffmpeg:
  camera:
    camera_one:
      name: Camera 1
      host: 192.168.XX.X
      port: 554
      path: /Streaming/Channels/101/
      username: !secret camera_one_user
      password: !secret camera_one_pass
      // highlight-start
      video_filters: # This makes sure the timestamp is added to the frames processed by Viseron (thumbnails, snapshots etc)
        - drawtext=text='%{localtime}':x=(w-tw):y=h-(2*lh):fontcolor=white:box=1:boxcolor=0x00000000@1:fontsize=20
      recorder:
        codec: h264 # Instruct FFmpeg to re-encode the video. This is needed to add the timestamp since video filters cannot be used with copy codec
        video_filters: # This makes sure the timestamp is added to the recorded videos
          - drawtext=text='%{localtime}':x=(w-tw):y=h-(2*lh):fontcolor=white:box=1:boxcolor=0x00000000@1:fontsize=20
      // highlight-end
```

</details>

### Raw command

If you want to use a custom FFmpeg command, you can do so by specifying the `raw_command` option.
Viseron needs to be able to read frames, so you must make sure to output frames to stdout.
By default this is done using the `pipe:1` as the output file.

You also need to make sure that you are outputting frames in the raw format (`-f rawvideo`) that Viseron expects.

The third consideration is that small segments need to be saved to disk for processing by the recorder.
This is done by using the format `-f segment`.

To get the hang of it you can start by using the default command and then modify it to your liking.

#### Raw command with substream

If you specify `substream`, Viseron will use the substream for motion/object detection, which means that the raw command for the substream has to output to `pipe:1`.<br />
The main stream raw command will be used for recordings, so it has to output segments using `-f segment`.

<details>
  <summary>Example using both main and substream</summary>

```yaml title="/config/config.yaml"
ffmpeg:
  camera:
    camera_one:
      name: Camera 1
      host: 192.168.XX.X
      port: 554
      path: /onvif_camera/profile.0
      username: !secret camera_one_user
      password: !secret camera_one_pass
      width: 1920
      height: 1080
      fps: 30
      substream:
        port: 554
        path: /onvif_camera/profile.1
        width: 1920
        height: 1080
        fps: 1
        raw_command: | # Output to pipe:1
          ffmpeg -rtsp_transport tcp -i rtsp://user:pass@192.168.XX.X:554/onvif_camera/profile.1 -vf fps=1.0 -f rawvideo -pix_fmt nv12 pipe:1
      raw_command: | # Output segments to /segments/camera_one
        ffmpeg -rtsp_transport tcp -i rtsp://user:pass@192.168.XX.X:554/onvif_camera/profile.0 -f segment -segment_time 5 -reset_timestamps 1 -strftime 1 -c:v copy /segments/camera_one/%Y%m%d%H%M%S.mp4
```

</details>

:::warning

Most of the configuration options are ignored when using `raw_command`.

:::

:::note

If you create a command that works well for your particular hardware, please share it with the community!

:::

<ComponentTroubleshooting meta={ComponentMetadata} />
